Impact of individual level uncertainty of lung cancer polygenic risk score (PRS) on risk stratification

Background Although polygenic risk score (PRS) has emerged as a promising tool for predicting cancer risk from genome-wide association studies (GWAS), the individual-level accuracy of lung cancer PRS and the extent to which its impact on subsequent clinical applications remains largely unexplored. Methods Lung cancer PRSs and confidence/credible interval (CI) were constructed using two statistical approaches for each individual: (1) the weighted sum of 16 GWAS-derived significant SNP loci and the CI through the bootstrapping method (PRS-16-CV) and (2) LDpred2 and the CI through posteriors sampling (PRS-Bayes), among 17,166 lung cancer cases and 12,894 controls with European ancestry from the International Lung Cancer Consortium. Individuals were classified into different genetic risk subgroups based on the relationship between their own PRS mean/PRS CI and the population level threshold. Results Considerable variances in PRS point estimates at the individual level were observed for both methods, with an average standard deviation (s.d.) of 0.12 for PRS-16-CV and a much larger s.d. of 0.88 for PRS-Bayes. Using PRS-16-CV, only 25.0% of individuals with PRS point estimates in the lowest decile of PRS and 16.8% in the highest decile have their entire 95% CI fully contained in the lowest and highest decile, respectively, while PRS-Bayes was unable to find any eligible individuals. Only 19% of the individuals were concordantly identified as having high genetic risk (> 90th percentile) using the two PRS estimators. An increased relative risk of lung cancer comparing the highest PRS percentile to the lowest was observed when taking the CI into account (OR = 2.73, 95% CI: 2.12–3.50, P-value = 4.13 × 10−15) compared to using PRS-16-CV mean (OR = 2.23, 95% CI: 1.99–2.49, P-value = 5.70 × 10−46). Improved risk prediction performance with higher AUC was consistently observed in individuals identified by PRS-16-CV CI, and the best performance was achieved by incorporating age, gender, and detailed smoking pack-years (AUC: 0.73, 95% CI = 0.72–0.74). Conclusions Lung cancer PRS estimates using different methods have modest correlations at the individual level, highlighting the importance of considering individual-level uncertainty when evaluating the practical utility of PRS. Supplementary Information The online version contains supplementary material available at 10.1186/s13073-024-01298-4.


Background
Lung cancer is a multifactorial disease with high incidence and mortality [1,2].Environmental exposures including tobacco smoking [3][4][5][6], occupational exposures [7,8], and air pollution [9][10][11], as well as heritable genetics, contribute to lung cancer risk [12].Unfortunately, a majority of lung cancer cases are diagnosed at a late disease stage with a poor 5-year survival rate of less than 5%.There is, therefore, an urgent and unmet need to detect lung cancer early when prevention or earlier intervention is possible [1].Polygenic risk scores (PRSs) have emerged as a promising tool for predicting cancer risk from genome-wide association studies (GWAS) [13][14][15][16].
As PRS prediction moves towards clinical implementation in personalized medicine, accurate and unbiased PRS predictions for any single individual are needed.Unfortunately, to date, the predictive accuracy of PRS has been mostly evaluated at the population level using cohort-level metrics of prediction R 2 and area under the curve (AUC), with its precision at a single individual level remaining largely unexplored [21,22].Moreover, whether individual-level PRS instability would influence the subsequent clinical utilization of PRS-based risk stratification and prediction is also of great interest to discover [21][22][23][24][25].In recent studies, the individual level uncertainty in PRS estimation on various traits including height and body mass index, and diseases including breast cancer, hypertension, and dementia have been assessed in the British population using UK Biobank data [21,22].Another study showed that post-traumatic stress disorder and type 2 diabetes PRSs estimated among populations from different ancestries have very modest correlations at the individual level [24].To our knowledge, there has been no study investigating the stability of lung cancer PRS for individuals and its potential impact on subsequent prediction for downstream clinical applications.
Using data from the International Lung Cancer Consortium (ILCCO) [16], we estimated lung cancer PRSs and constructed corresponding confidence/credible interval (CI) for each individual using two statistical approaches: (1) the weighted sum of 16 GWAS-derived significant single nucleotide polymorphisms (SNP) loci that have been validated in European descent population and the CI through bootstrapping method (PRS-16-CV) and (2) LDpred2 and the CI through posteriors sampling (PRS-Bayes) [21].We further evaluated the impact of individual-level PRS uncertainty on PRS-based ranking, risk stratification, and prediction in conjunction with non-genetic risk factors of age, gender, and smoking history.Our research shows that the uncertainty of lung cancer PRS at the individual level greatly impacts the subsequent performance of individual risk stratification and prediction, highlighting the importance of cautious clinical interpretation and implementation in precision medicine.

Study population
We conducted our study in 30,060 participants of European ancestry (17,166 lung cancer cases and 12,894 controls) from 25 lung cancer OncoArray studies of ILCCO including both population-based and hospital-based case-control studies.The basic characteristics of these studies are summarized in Table S1 (Fig. 1, Additional file 1: Table S1).The eligibility criteria of studies to be included in ILCCO were that they had a study protocol for subject recruitment and a structured questionnaire for baseline lifestyle information.A detailed description of each study was described in the consortium flagship paper [16,26].Baseline demographics of age, gender, self-reported race/ethnicity, smoking history, and lung cancer histology information were collected and adjusted for in multivariable logistic regression analyses.

Genotyping, imputation, and quality control
Genotyping was performed at the Center for Inherited Disease Research using the OncoArray platform with 533,631 SNPs from Illumina and imputation was conducted based on 1000G phase 3 reference panel [27].Standard QC procedures were applied to keep variants with genotype calling rate > 95%, minor allele frequency > 1%, and deviation from Hardy-Weinberg equilibrium with P-value > 10 −10 in controls.A total of 5,097,871 variants remained in our statistical analyses.To adjust for subtle population stratification, we performed a principal component analysis using PLINK v1.9 [28] and adopted the top 10 principal components (PCs) following reference [29].

PRS estimation and confidence/credible interval (CI) construction
PRS-16-CV was calculated as the weighted sum of 16 GWAS-derived common SNP loci that have been validated in European descent populations [18].Effect sizes were estimated through five-fold cross-validation after Fig. 1 Overview of the study.The study was conducted in 17,166 lung cancer cases and 12,894 controls with European ancestry from the International Lung Cancer Consortium (ILCCO).The lung cancer PRSs and corresponding confidence/credible interval were constructed using two statistical approaches for each individual-(1) the weighted sum of 16 GWAS-derived significant SNP loci that have been validated in European descent population and the confidence interval through the bootstrapping method (PRS-16-CV) and (2) LDpred2 and the credible interval through posteriors sampling (PRS-Bayes).The individual-level PRS uncertainty was characterized and the impact on subsequent risk stratification and prediction were evaluated adjusting for age, gender, smoking status, and 10 PCs.Confidence intervals (CI) for PRS-16-CV were constructed using 1000 bootstrap samples within each fold.Therefore, each individual would have their own individual-level PRS distribution characterized by the PRS-16-CV bootstrapped mean and CI for downstream risk stratification and prediction.Cross-validation and bootstrapping were implemented using R v4.1.0.In addition, we calculated the PRS-16 based on the same 16 SNP loci with effect sizes directly extracted from the GWAS catalog and previous literature (Additional file 1: Table S2-3), and no confidence interval was provided for the PRS-16 [16,18,[30][31][32][33].
On the other hand, PRS-Bayes was estimated using a Bayesian approach leveraging the LDpred2 framework, and PRS-Bayes credible interval was constructed via posterior distribution sampling using the method proposed by Ding et al. [21,34].More specifically, we utilized the same training data in PRS-16-CV to estimate a posterior distribution of effect sizes, given the observed genotype and lung cancer status.For each variant, a sample of effect size estimates was drawn from the posterior distribution of the causal effect size using Markov chain Monte Carlo.A credible interval of the PRS-Bayes estimator was constructed by aggregating the number of effect allele copies weighted by each of the drawn estimates.

Characterization of PRS uncertainty at the individual level
We characterized the individual level uncertainty in PRS estimation with standard deviation (s.d.) and a pre-specified confidence/credible level of CI.With a pre-specified confidence/credible level p, we derived an individual p-level CI of PRS by obtaining the empirical (1-p)/2 and (1+p)/2 quantiles from the individual level PRS distribution.

Impact of individual-level PRS uncertainty on PRS-based risk stratification
Having obtained PRSs and corresponding CI, we stratified individuals into different PRS risk subgroups based on the relationship between their own PRS CI and the population level threshold (Fig. 2).Individuals with their PRS CI above a pre-specified population-level threshold t at the upper tail (e.g., t = 90th percentile) were classified as certainly high genetic risk and similarly for individuals with PRS CI below the population level threshold t at the lower tail (e.g., t = 10th percentile) as certainly low genetic risk.Individuals whose CI covered the population level threshold were considered uncertain.As a comparison, we classified individuals based on their estimated PRS mean and population threshold without taking individual-level certainty into account.
To evaluate the degree of consistency for PRS-based risk stratification, we counted the number of overlapped individuals that are concordantly identified as the same PRS-based risk by different PRS estimators.Sensitivity analyses were conducted by changing the confidence/ credible level p and risk threshold t at the population level.In addition to the two PRS approaches, we also assessed the concordance of PRS-based risk stratification among 22 lung cancer PRS in the PGS catalog.

Impact of individual level PRS uncertainty on relative risk and risk prediction
We applied multivariable logistic regression to discover the effect of PRS-based risk subgroups on lung cancer risk controlling for age, gender, and smoking history in the individuals who were identified with certainty.Stratified analyses by gender, smoking status, and lung cancer histology were similarly conducted.Lung cancer risk prediction models were constructed by integrating both PRS risk groups and other non-genetic baseline covariates, such as age, gender, and smoking history.The prediction model performance was evaluated using five-fold crossvalidation based on the metric of AUC.To investigate the potential impact of individual-level uncertainty on risk prediction, we constructed a risk prediction model within the individuals identified with certainty (certainly high vs. low risk) and compared the model performance constructed in the high (n = 3006) and low (n = 3006) risk deciles identified by PRS-16-CV mean without taking individual level uncertainty into account under the exact same model specification.

Results
Table 1 summarizes the baseline characteristics of 30,060 participants that were included in the study.Ever smokers were significantly enriched in lung cancer cases with a significantly longer median smoking pack-years (39 pack-years) compared to controls (13 pack-years).Among lung cancer patients, 76.4% of the cases were non-small cell lung cancer (NSCLC) with an enrichment of lung adenocarcinoma (38.1%).Three PRSs (PRS-16, PRS-16-CV, and PRS-Bayes) of lung cancer risk were calculated for each individual and statistically higher PRS mean (mPRS) were consistently observed in lung cancer patients compared to controls (mPRS-16 in lung cancer cases: 1.26, controls: 1.21, P-value < 2.2e−16; mPRS-16-CV in lung cancer cases: 1.21, controls: 1.16, P-value < 2.2e−16; mPRS-Bayes in lung cancer cases: − 0.04, controls: − 0.11, P-value < 2.2e−16), suggesting a potentially higher genetic risk in lung cancer patients (Additional file 2: Fig. S1).

Characterization of lung cancer PRS uncertainty at the individual level
The variability of lung cancer PRS at the individual level was evaluated by leveraging the CIs constructed in PRS-16-CV and PRS-Bayes.A considerable variation of PRS point estimates was observed for both methods.On average, a larger standard deviation (s.d.) of individual-level PRS distribution was observed in PRS-Bayes (mean s.d.: 0.88, 95% CI of s.d.: 0.68-1.11)compared to PRS-16-CV (mean s.d.: 0.12, 95% CI of s.d.: 0.09-0.15),suggesting a larger variability using PRS-Bayes at the individual level.For illustration purposes, we show the individual-level PRS distribution for both methods in 100 individuals (Fig. 3).

Impact of Individual-level PRS uncertainty on lung cancer risk stratification
All individuals were stratified into PRS deciles and at a confidence/credible level of 95%.Using PRS-16-CV, only 25.0% (751/3006) of individuals with PRS point estimates in the lowest decile of PRS and 16.8% (505/3006) in the highest decile have their entire 95% CI fully contained in the lowest and highest decile, respectively, while PRS-Bayes was unable to find any eligible individual (Table 2).
We further conducted sensitivity analysis by changing the population level threshold t and the confidence/credible level p.We varied the range of the confidence/credible level p from 0 to 100% and fixed the threshold at t = 10th or 5th percentile for the low-risk population and at t = 90th or 95th percentile for the high-risk population.The proportion of certainty was negatively correlated with the confidence/credible level p for both the highrisk and low-risk classification (Additional file 2: Fig. S2).
A consistently lower percentage of certain classifications was observed in PRS-Bayes than in PRS-16-CV across all confidence/credible levels.Similar relationships between the proportion and the confidence/credible level p were observed across subgroups of gender, histology, and smoking status (Additional file 2: Fig. S3).The proportion of certainty decreases as more stringent threshold t and confidence/credible level p are specified.Furthermore, the individual-level uncertainty greatly impacts PRS-based rankings.Each individual would have a distribution of the rankings for each PRS point estimate obtained from both PRS-16-CV and PRS-Bayes.
We calculated the mean and range of the rankings for each individual.Substantial variability was also observed in the rankings identified by both methods for each PRS decile (Table 2).A wider range of rankings was observed for PRS-Bayes compared to using PRS-16-CV.The minimal range of the rankings was observed in the lowest decile (mean ranking 7th, range: 0-66th) identified by

Table 2 PRS-based rankings identified by PRS-16-CV and PRS-Bayes
The mean and range of the rankings for each individual within each decile identified by PRS-16-CV and PRS-Bayes were calculated.The column n indicates the number of individuals that can be identified with certainty.Using PRS-16-CV, 751 individuals in the lowest PRS decile and 505 individuals in the highest decile can be identified with certainty.In contrast, PRS-Bayes were not able to identify any individuals with certainty

Mean rankings [range] n Mean rankings [range]
0th-10th PRS-16-CV, and individuals above the 90th percentile of PRS can be anywhere from the 25th-100th percentile.In contrast, individuals in each PRS decile can be anywhere from the 0th-100th using PRS-Bayes when their CI is taken into consideration.
To answer the question of whether individuals would be commonly assigned into the same genetic risk categories, we counted the number of overlapped individuals that were commonly identified using different PRS estimators of PRS-16, PRS-16-CV, and PRS-Bayes (Table 3).Overall, the degree of overlap decreases as the population level threshold increases.PRS-16 and PRS-16-CV identified a fair number of overlapped individuals as the same SNP loci were utilized.Two thousand four hundred seventy (82%) individuals were concordantly classified as high risk (> 90th percentile) using PRS-16 and PRS-16-CV, while PRS-Bayes agreed with either PRS-16-CV or PRS-16 on 19% of high-risk (> 90th percentile) stratification.In addition, we assessed the concordance of 22 lung cancer PGSs available in the PGS catalog, and similar modest correlations in PRS-based risk stratification were observed using PGS estimate (Additional file 2: Fig. S4-7).Taken together, substantial disagreement was observed for lung cancer PRS at the individual level using different PRS estimators.

Impact of individual-level uncertainty on relative risk of PRS deciles on lung cancer
Next, we evaluated its impact on lung cancer risk in individuals identified by PRS-16-CV taking individual-level uncertainty into account.In contrast to risk stratification based on PRS mean, only individuals in the top and lowest deciles were able to be identified with certainty when taking variance in PRS estimates into account; hence, the relative risk was only evaluated in the two PRS-based risk subgroups.An increased effect size for PRS CI (OR = 2.73, 95% CI: 2.12-3.50,P-value = 4.13 × 10 −15 ) was compared to using PRS mean (OR = 2.23, 95% CI: 1.99-2.49,P-value = 5.70 × 10 −46 ) (Table 4).Similar improvement was observed in stratified analyses by gender, lung cancer histology, and smoking status (Table 5).The largest increase in relative risk of lung cancer from OR = 2.63 to OR = 4.22 was observed in never smokers, suggesting a potentially larger genetic contribution to lung cancer and thus more susceptible to individual-level PRS uncertainty.

Impact of individual-level PRS uncertainty on lung cancer risk prediction
Finally, we evaluated the impact of the individual-level PRS uncertainty on lung cancer risk prediction by

Discussion
Our work explored lung cancer PRS uncertainty for individuals in populations with European ancestry using both GWAS-derived SNPs and LDpred2 and demonstrated that individual-level PRS uncertainty greatly impacts PRS-based rankings, risk stratification, and ultimately prediction of lung cancer risk.The substantial recent interest in translating lung cancer PRS and other predictive tools like deep learning models from low radiation dose chest computed tomography (LDCT) for future lung cancer risk prediction necessitates a careful assessment of individual-level uncertainty to truly accomplish personalized risk assessment [35,36].Taking individuallevel uncertainty into PRS-based risk stratification and prediction provides more accurate risk stratification and improves the ability to identify high-risk subjects and recommend LDCT screening with more certainty.It is imperative to develop more stable PRS and account for uncertainty at the individual level in risk stratification to Table 6 Impact of individual-level uncertainty on lung cancer risk prediction performance The prediction model performances incorporating different risk factors of PRS-based risk subgroup, age, gender, and smoking history were evaluated in subsets of individuals that were identified by the PRS CI-based approach and by the PRS mean-based approach.For the PRS CI-based approach, the models were constructed and evaluated in the individuals that can be identified with certainty (n = 1256).As a comparison, we constructed the same models and evaluated in the individuals that were classified as the lowest and highest risk by PRS-16-CV mean (n = 6012).The model performance was evaluated using five-fold cross-validation.Area under the curve (AUC) and 95% confidence intervals (CI) are shown A critical concern of PRS application is delivering inaccurate risk estimates at the individual level, and wrongly categorizing an individual as low or high genetic risk based on unstable PRS estimates.The downstream effect could be inappropriate or even contradictory medical advice or clinical decisions [25].Our study shows substantial disagreement in risk categorization using different lung cancer PRSs (PRS-16, PRS-16-CV, and PRS-Bayes), suggesting that individuals that were identified as very high risk by one PRS method may not be classified as such by another.Similar modest correlations were observed for another 22 lung cancer PRS in the PGS catalog.This issue is not unique to lung cancer, as large discordances have also been found in breast cancer, hypertension, and dementia using approaches to calculate PRS in a white British population [22].More complex traits or diseases and different ancestries between the discovery and target population of GWAS may lead to even more profound inconsistency at the individual level [24].PRS needs to be reliable and reproducible if it is going to inform personalized decision-making in clinical settings.
Comparing the two PRS generative methods, a much larger variability was observed in PRS-Bayes, resulting in no individuals can be identified with sufficient certainty.Ding et al. also found only a limited proportion of individuals are classified as high risk with certainty across 13 traits in UK Biobank [21].PRS-Bayes includes a much greater number of non-zero weight SNPs (~2000 SNPs) with small effect sizes and this may explain a higher proportion of SNP-based heritability compared to PRS-16-CV (16 SNPs).In the meanwhile, the numerous SNPs with small effect sizes may contribute to a higher variance in PRS estimates.PRS-Bayes may perform better in terms of lung cancer risk prediction at the population level as the accuracy is determined by the proportion of phenotypic variance explained by variants included and the improved population prediction error.On the other hand, PRS-16-CV is relatively parsimonious and contains only those that have been robustly validated SNPs and have a minor individual-level variance.Additionally, we constructed and evaluated the individual-level PRS uncertainty and population-level prediction accuracy using nine experimentally validated SNPs that are associated with lung cancer risk [37] (Additional file 1: Table S4).More individuals can be identified with certainty while similar prediction accuracy was achieved (Additional file 1: Table S5-6).This may suggest that experimentally validated fine-mapped variants may be more likely to be biologically causal variants and thus have stable effects and PRS risk stratification performance at the individual level for lung cancer prediction given the similar population-level prediction accuracy.A tradeoff between these methods may be manifested by the compromise between prediction performance at the population level and stability at the individual level, and this needs to be considered cautiously when developing new PRS methods tailoring for different purposes and applications [21].
Given the individual-level PRS uncertainty, it is imperative to cautiously interpret and implement it in clinical applications.From our analyses, using the PRS-based risk subgroup alone resulted in the lowest AUC, and including other non-genetic factors of age, gender, and smoking history largely improved the lung cancer risk prediction performance.PRS stand-alone is imperfect and the use of PRS as a covariate or effect modifier in conjunction with non-genetic risk factors may inform better outcomes [22,38].Despite modest improvement in prediction accuracy using PRS-based risk subgroups taking individual-level uncertainty into account, likely due to the limited sample sizes of eligible individuals, it attains both PRS stability and predictive accuracy given the data.Also, it reflects the situation that it is challenging for the current PRS approaches to balance reproducible PRS for individuals with sufficient certainty and high prediction accuracy at the population level.It is crucial and pressing to develop guidelines in constructing PRS to minimize the extent to which individuals could be provided with imprecise or contradictory clinical advice and intervention as well as PRS reporting standards in light of the patient and primary care providers' perspective [25].
There are several improvements and future directions for the present study.First, more statistical approaches to construct lung cancer PRS, e.g., regularization-based methods, can be included [39].Second, we did not assess the potential interactions between lung cancer PRS and non-genetic risk factors when taking individual-level uncertainty into account, and we would expect more uncertainty arising from environmental factors as well as from the gene-environment interactions in predicting the risk of lung cancer [40].Further research is needed to assess the impact on performance when using commonly applied LD clumping methods for SNP selection, particularly with varying parameters.Lastly, our study was conducted in a population of European ancestry; thus, the generalizability of the results in other populations of different genetic ancestries is a concern given the discrepancy of genetic structures for causal variants, allele frequencies, and LD patterns across ancestries.To realize the equitable and reliable potential of PRS, it is necessary to carry out larger, multi-ancestry GWAS and further investigate the uncertainty at the individual level in PRS.

Conclusions
Our study characterized the uncertainty of lung cancer PRS at the individual level and evaluated its impact on subsequent PRS-based ranking, risk stratification, and prediction in populations with European ancestry.It is imperative to develop reproducible PRS, reliable PRS-based clinical applications as well as guidelines to report and communicate PRS to patients and their primary physicians.

Fig. 2
Fig. 2 Risk stratification based on PRS confidence/credible interval (CI).The large distribution illustrates the PRS distribution at the population level, and the four small ones refer to individual PRS distributions for participants with different PRS-based risks of lung cancer.The dashed horizontal lines indicate the population level thresholds for risk stratification.Individuals with their PRS CI above a pre-specified population-level threshold t at the upper tail (e.g., t = 90th percentile) were classified as certainly high genetic risk, and similarly for individuals with PRS CI below the population level threshold t at the lower tail (e.g., t = 10th percentile) as certainly low genetic risk.Individuals whose CI covered the population level threshold were considered uncertain

Fig. 3
Fig. 3 Individual-level distribution of PRS-16-CV and PRS-Bayes.A Individual-level PRS distributions obtained from PRS-16-CV.B Individual-level PRS distributions obtained from PRS-Bayes.For illustration purposes, here we only show the individual-level PRS distributions for 100 participants

Table 1
Baseline demographic information of the study population SD standard deviation; NSCLC non-small cell lung cancer; ADC adenocarcinoma, SCC squamous cell carcinoma; SCLC small cell lung

Table 3
Overlap of commonly identified individuals using different PRS estimatorsThe number and percentage of individuals that can be commonly identified with the same PRS-based risk groups by any of the two PRS estimators (PRS-16 vs. PRS-16-CV; PRS-16 vs. PRS-Bayes; PRS-16-CV vs. PRS-Bayes) were shown as the threshold of PRS risk increases

Table 4
Impact of individual-level uncertainty on relative risk of PRS deciles on lung cancer risk Odds ratio of lung cancer comparing different PRS deciles identified by PRS CI-based approach taking individual level uncertainty into account and by PRS mean were shown.As the PRS-16-CV CI-based approach was only able to identify individuals in the lowest (n = 751) and highest decile (n = 505) with certainty, the analysis was only conducted in the two subsets.In contrast, using the PRS-16-CV mean, the analyses were conducted in each PRS decile compared to the lowest one.The detailed sample sizes that were included in each analysis were noted in column n.Odds ratio (OR) and 95% confidence intervals (CI) are shown.All models were adjusted for age, gender, and smoking status

Table 5
Impact of individual-level uncertainty on the relative risk of PRS deciles on lung cancer risk in stratified analyses by gender, lung cancer histology, and smoking statusFor the PRS CI-based approach, the stratified analyses were only conducted in the individuals that can be identified with certainty.The detailed sample sizes that were included in each analysis were noted in column n.Odds ratio (OR) and 95% confidence intervals (CI) are shown.NSCLC, non-small cell lung cancer; ADC, adenocarcinoma, SCC, squamous cell carcinoma; SCLC, small cell lung cancer